The European Organisation of External Quality Assurance Providers in Laboratory Medicine (EQALM) Statement: guidelines for publishing about interlaboratory comparison studies (PubILC).

Data and results from interlaboratory comparison (ILC) studies, external quality assessment (EQA) and proficiency testing (PT) activities are important and valuable contributions both to the further development of all disciplines of medical laboratory diagnostics, and to the evaluation and comparison of in vitro diagnostic assays. So far, however, there are no recommendations as to which essential items should be addressed in publications on interlaboratory comparisons. The European Organization of External Quality Assurance Providers in Laboratory Medicine (EQALM) recognized the need for such recommendations, and these were developed by a group of experts. The result of this endeavor is the EQALM Statement on items recommended to be addressed in publications on interlaboratory comparison activities (PubILC), in conjunction with a user-friendly checklist. Once adopted by authors and journals, the EQALM Statement will ensure essential information and/or study-related facts are included within publications on EQA/PT activities.

External quality assurance in the era of standardization.

Metrology in clinical chemistry aims to ensure the equivalence of measurement results from different in-vitro diagnostic measurement devices (IVD MD) for use in healthcare. The metrological traceability of measurement results to higher-order references is the cornerstone to achieving equivalent results. However, other fundamentals are also needed, including the commutability of reference materials and external quality assessment (EQA) materials for monitoring the equivalence of measurement results at the end-user level. This manuscript summarizes the findings and opinions expressed at the Joint Community for Traceability in Laboratory Medicine (JCTLM) workshop held on December 4-5, 2023. The workshop explored the relationship between EQA/proficiency testing and metrological traceability to higher-order references. EQA monitors the equivalence of measurement results from end-user IVD MDs. The workshop discussed the role and challenges of using EQA to improve and maintain the equivalence of measurement results. It also elucidated current developments in establishing the clinical suitability of laboratory results expressed as analytical performance specifications (APS).

Sharing reference intervals and monitoring patients across laboratories - findings from a likely commutable external quality assurance program.

OBJECTIVES: Laboratory results are increasingly interpreted against common reference intervals (CRIs), published clinical decision limits, or previous results for the same patient performed at different laboratories. However, there are no established systems to determine whether current analytical performance justifies these interpretations. We analysed data from a likely commutable external quality assurance program (EQA) to assess these interpretations. METHODS: The use of CRIs was assessed by evaluating instrument group medians against minimum specifications for bias. The use of clinical decision limits was assessed using specifications from professional bodies, and the monitoring of patients by testing at different laboratories was assessed by comparing all-laboratory imprecision to within-subject biological variation. RESULTS: Five of the 18 analytes with Australasian CRIs did not meet specification for all instrument groups. Among these, calcium and magnesium failed for one instrument group out of seven, while bicarbonate, chloride, and lipase failed for two instrument groups. Of the 18 analytes reviewed currently without CRIs in Australasia, 10 candidates were identified. Among analytes with clinical decision limits, i.e. lipids, glucose, and vitamin D, only triglycerides met both bias and imprecision specifications, while vitamin D met the imprecision specification. Monitoring patients by testing at different laboratories was supported for 15 of the 46 (33 %) analyte-method principles groups that met minimum imprecision specifications. CONCLUSIONS: Analysis of data from commutable EQA programs can provide a mechanism for monitoring whether analytical performance justifies the interpretations made in contemporary laboratory practice. EQA providers should establish systems for routinely providing this information to the laboratory community.

Investigating the Current Harmonization Status of Tumor Markers Using Global External Quality Assessment Programs: A Feasibility Study.

BACKGROUND: The harmonization status of most tumor markers (TMs) is unknown. We report a feasibility study performed to determine whether external quality assessment (EQA) programs can be used to obtain insights into the current harmonization status of the tumor markers α-fetoprotein (AFP), prostate specific antigen (PSA), carcinoembryonic antigen (CEA), cancer antigen (CA)125, CA15-3 and CA19-9. METHODS: EQA sample results provided by 6 EQA providers (INSTAND [Germany], Korean Association of External Quality Assessment Service [KEQAS, South Korea], National Center for Clinical Laboratories [NCCL, China], United Kingdom National External Quality Assessment Service [UK NEQAS, United Kingdom], Stichting Kwaliteitsbewaking Medische Laboratoriumdiagnostiek [SKML, the Netherlands], and the Royal College of Pathologists of Australasia Quality Assurance Programs [RCPAQAP, Australia]) between 2020 and 2021 were used. The consensus means, calculated from the measurement procedures present in all EQA programs (Abbott Alinity, Beckman Coulter DxI, Roche Cobas, and Siemens Atellica), was used as reference values. Per measurement procedure, the relative difference between consensus mean for each EQA sample and the mean of all patient-pool-based EQA samples were calculated and compared to minimum, desirable, and optimal allowable bias criteria based on biological variation. RESULTS: Between 19040 (CA15-3) and 25398 (PSA) individual results and 56 (PSA) to 76 (AFP) unique EQA samples were included in the final analysis. The mean differences with the consensus mean of patient-pool-based EQA samples for all measurement procedures were within the optimum bias criterion for AFP, the desirable bias for PSA, and the minimum bias criterion for CEA. However, CEA results <8 µg/L exceeded the minimum bias criterion. For CA125, CA15-3, and CA19-9, the harmonization status was outside the minimum bias criterion, with systematic differences identified. CONCLUSIONS: This study provides relevant information about the current harmonization status of 6 tumor markers. A pilot harmonization investigation for CEA, CA125, CA15-3, and CA19-9 would be desirable.

A model for managing quality control for a network of clinical chemistry instruments measuring the same analyte.

OBJECTIVES: Monitoring quality control for a laboratory or network with multiple instruments measuring the same analyte is challenging. We present a retrospective assessment of a method to detect medically significant out-of-control error conditions across a group of instruments measuring the same analyte. The purpose of the model was to ensure that results from any of several instruments measuring the same analytes in a laboratory or a network of laboratories provide comparable results and reduce patient risk. Limited literature has described how to manage QC in these very common situations. METHODS: Single Levey-Jennings control charts were designed using peer group target mean and control limits for five common clinical chemistry analytes in a network of eight analyzers in two different geographical sites. The QC rules used were 13s/22s/R4s, with the mean being a peer group mean derived from a large population of the same instrument and the same QC batch mean and a group CV. The peer group data used to set the target means and limits were from a quality assurance program supplied by the instrument supplier. Both statistical and clinical assessments of significance were used to evaluate QC failure. Instrument bias was continually monitored. RESULTS: It was demonstrated that the biases of each instrument were not statistically or clinically different compared to the peer group's average over six months from February 2023 until July 2023. Over this period, the error rate determined by the QC model was consistent with statistical expectations for the 13s/22s/R4s rule. There were no external quality assurance failures, and no detected error exceeded the TEa (medical impact). Thus, the combined statistical/clinical assessment reduced unnecessary recalibrations and the need to amend results. CONCLUSIONS: This paper describes the successful implementation of a quality control model for monitoring a network of instruments, measuring the same analytes and using externally provided quality control targets. The model continually assesses individual instrument bias and imprecision while ensuring all instruments in the network meet clinical goals for quality. The focus of this approach is on detecting medically significant out-of-control error conditions.

Advances in clinical chemistry patient-based real-time quality control (PBRTQC).

Patient-Based Real-Time Quality Control involves monitoring an assay using patient samples rather than external material. If the patient population does not change, then a shift in the long-term assay population results represents the introduction of a change in the assay. The advantages of this approach are that the sample(s) are commutable, it is inexpensive, the rules are simple to interpret and there is virtually continuous monitoring of the assay. The disadvantages are that the laboratory needs to understand their patient population and how they may change during the day, week or year and the initial change of mindset required to adopt the system. The concept is not new, having been used since the 1960s and widely adopted on hematology analyzers in the mid-1970s. It was not widely used in clinical chemistry as there were other stable quality control materials available. However, the limitations of conventional quality control approaches have become more evident. There is a greater understanding of how to collect and use patient data in real time and a range of powerful algorithms which can identify changes in assays. There are more assays on more samples being run. There is also a greater interest in providing a theoretical basis for the validation and integration of these techniques into routine practice.

Methods for determining clinical utility.

Measuring the clinical utility of a diagnostic test involves evaluating its impact on patient outcomes, clinical decision-making, and healthcare resource utilization. Determining clinical utility requires accessing patient medical history and outcomes data. These studies involve enrolling patients undergoing diagnostic tests and tracking their clinical outcomes. Researchers can determine the test's clinical utility by comparing the outcomes of patients who receive the diagnostic test to those who do not. These outcomes include benefits and harm. The highest level of evidence to support clinical utility determinations may be obtained from clinical trials. However, clinical laboratories are often not involved in clinical trials, and laboratory specialists may not be experienced in conducting such trials. Many established laboratory tests have never had clinical utility determined. Prospective studies assessing a diagnostic test's impact on clinical outcomes may require long-term patient monitoring, which is problematic. This paper presents methods that may be used to assess clinical utility.

Clinical utility - Information about the usefulness of tests.

The clinical utility of a diagnostic test refers to its usefulness in improving patient outcomes, informing clinical decision-making, and optimizing healthcare resources. A diagnostic test with high clinical utility provides accurate, reliable, and actionable information that can guide appropriate treatment decisions, monitor treatment response, and identify potential adverse events or complications. Ultimately, the clinical utility of a diagnostic test depends on how well it can improve patient outcomes by guiding appropriate treatment decisions, improving clinical outcomes, and optimizing healthcare resource utilization. Healthcare providers need to weigh the benefits and drawbacks of using a particular diagnostic test in their clinical practice to determine its clinical utility.

Classification of "Near-patient" and "Point-of-Care" SARS-CoV-2 Nucleic Acid Amplification Test Systems and a first approach to evaluate their analytical independence of operator activities.

BACKGROUND: European legislation defines as "near-patient testing" (NPT) what is popularly and in other legislations specified as "point-of-care testing" (POCT). Systems intended for NPT/POCT use must be characterized by independence from operator activities during the analytic procedure. However, tools for evaluating this are lacking. We hypothesized that the variability of measurement results obtained from identical samples with a larger number of identical devices by different operators, expressed as the method-specific reproducibility of measurement results reported in External Quality Assessment (EQA) schemes, is an indicator for this characteristic. MATERIALS AND METHODS: Legal frameworks in the EU, the USA and Australia were evaluated about their requirements for NPT/POCT. EQA reproducibility of seven SARS-CoV-2-NAAT systems, all but one designated as "POCT", was calculated from variabilities in Ct values obtained from the respective device types in three different EQA schemes for virus genome detection. RESULTS: A matrix for characterizing test systems based on their technical complexity and the required operator competence was derived from requirements of the European In Vitro Diagnostic Regulation (IVDR) 2017/746. Good EQA reproducibility of the measurement results of the test systems investigated implies that different users in different locations have no recognizable influence on their measurement results. CONCLUSION: The fundamental suitability of test systems for NPT/POCT use according to IVDR can be easily verified using the evaluation matrix presented. EQA reproducibility is a specific characteristic indicating independence from operator activities of NPT/POCT assays. EQA reproducibility of other systems than those investigated here remains to be determined.

Machine Learning in Laboratory Medicine: Recommendations of the IFCC Working Group.

BACKGROUND: Machine learning (ML) has been applied to an increasing number of predictive problems in laboratory medicine, and published work to date suggests that it has tremendous potential for clinical applications. However, a number of groups have noted the potential pitfalls associated with this work, particularly if certain details of the development and validation pipelines are not carefully controlled. METHODS: To address these pitfalls and other specific challenges when applying machine learning in a laboratory medicine setting, a working group of the International Federation for Clinical Chemistry and Laboratory Medicine was convened to provide a guidance document for this domain. RESULTS: This manuscript represents consensus recommendations for best practices from that committee, with the goal of improving the quality of developed and published ML models designed for use in clinical laboratories. CONCLUSIONS: The committee believes that implementation of these best practices will improve the quality and reproducibility of machine learning utilized in laboratory medicine. SUMMARY: We have provided our consensus assessment of a number of important practices that are required to ensure that valid, reproducible machine learning (ML) models can be applied to address operational and diagnostic questions in the clinical laboratory. These practices span all phases of model development, from problem formulation through predictive implementation. Although it is not possible to exhaustively discuss every potential pitfall in ML workflows, we believe that our current guidelines capture best practices for avoiding the most common and potentially dangerous errors in this important emerging field.

Implementation of circulating tumour DNA multi-target mutation testing in plasma: a perspective from an external quality assessment providers' survey.

Demand for large-scale tumour profiling across cancer types has increased in recent years, driven by the emergence of targeted drug therapies. Analysing alternations in plasma circulating tumour DNA (ctDNA) for cancer detection can improve survival; ctDNA testing is recommended when tumour tissue is unavailable. An online survey of molecular pathology testing was circulated by six external quality assessment members of IQN Path to registered laboratories and all IQN Path collaborative corporate members. Data from 275 laboratories across 45 countries were collected; 245 (89%) perform molecular pathology testing, including 177 (64%) which perform plasma ctDNA diagnostic service testing. The most common tests were next-generation sequencing-based (n = 113). Genes with known stratified treatment options, including KRAS (n = 97), NRAS (n = 84), and EGFR (n = 130), were common targets. The uptake of ctDNA plasma testing and plans to implement further testing demonstrates the importance of support from a well-designed EQA scheme.

Between and within calibration variation: implications for internal quality control rules.

The variability between calibrations can be larger than the within calibration variation for some measurement procedures, that is a large CVbetween:CVwithin ratio. In this study, we examined the false rejection rate and probability of bias detection of quality control (QC) rules at varying calibration CVbetween:CVwithin ratios. Historical QC data for six representative routine clinical chemistry serum measurement procedures (calcium, creatinine, aspartate aminotransferase, thyrotrophin, prostate specific antigen and gentamicin) were extracted to derive the CVbetween:CVwithin ratios using analysis of variance. Additionally, the false rejection rate and probability of bias detection of three 'Westgard' QC rules (2:2S, 4:1S, 10X) at varying CVbetween:CVwithin ratios (0.1-10), magnitudes of bias, and QC events per calibration (5-80) were examined through simulation modelling. The CVbetween:CVwithin ratios for the six routine measurement procedures ranged from 1.1 to 34.5. With ratios >3, false rejection rates were generally above 10%. Similarly for QC rules involving a greater number of consecutive results, false rejection rates increased with increasing ratios, while all rules achieved maximum bias detection. Laboratories should avoid the 2:2S, 4:1S and 10X QC rules when calibration CVbetween:CVwithin ratios are elevated, particularly for those measurement procedures with a higher number of QC events per calibration.

Developing an evidence-based approach to quality control.

Effective Quality Control remains one of the pillars of Clinical Biochemistry. An understanding of the possible analytical errors that may occur, how to detect them efficiently and how to prevent them from causing patient harm are critical components of a Quality System. For some time, there have been questions about the theoretical basis of the models used to describe and detect analytical error. The current theory recognises two types of error, systematic and random and a system based on sampling the analytical process using a synthetic material to detect these errors. However, there are at least two other errors that are present. One is related to the QC material and the other, irregular errors. In this Opinion Piece, some of the underlying assumptions of Quality Control systems are described and analysed.

The next wave of innovation in laboratory automation: systems for auto-verification, quality control and specimen quality assurance.

Laboratory automation in clinical laboratories has made enormous differences in patient outcomes, with a wide range of tests now available that are accurate and have a rapid turnaround. Total laboratory automation (TLA) has mechanised tube handling, sample preparation and storage in general chemistry, immunoassay, haematology, and microbiology and removed most of the tedious tasks involved in those processes. However, there are still many tasks that must be performed by humans who monitor the automation lines. We are seeing an increase in the complexity of the automated laboratory through further platform consolidation and expansion of the reach of molecular genetics into the core laboratory space. This will likely require rapid implementation of enhanced real time quality control measures and these solutions will generate a significantly greater number of failure flags. To capitalise on the benefits that an improved quality control process can deliver, it will be important to ensure that an automation process is implemented simultaneously with enhanced, real time quality control measures and auto-verification of patient samples in middleware. Therefore, it appears that the best solution may be to automate those critical decisions that still require human intervention and therefore include quality control as an integral part of total laboratory automation.